This specification relates to generating feature embeddings from a co-occurrence matrix.
In order to create word embeddings, systems attempt to compress the distributional structure of raw language co-occurrence statistics to generate compact representations that retain properties of the original space. For example, systems may implement matrix factorization models or sample from a sliding window. The created word embeddings can be used in natural language tasks such as determining semantic similarity, parsing and translation.